Neural Prediction of the User's Mood from Visual Input

Affect-adaptive systems mutate their behavior according to the user’s affective state. In many cases, such affective state is to be detected in a nonobtrusive way, i.e. through sensing that does not require the user to provide the system explicit input, e.g., video sensors. However, user affect recognition from video is frequently tuned to detect instantaneous emotional states, rather than longer term and more constant affective states such as mood. In this paper, we propose a non-linear computational model for bridging the gap between the recognized emotions of a person captured by a video and the overall mood of the person. For the experimental validation, emotions and mood are human annotations on an affective visual database that we created on purpose. Based on features describing peculiarities and changes in the user’s emotional state, our system is able to predict the corresponding mood well above chance and more accurately than existing models.